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Background/context: 

Description of prior research, its intellectual context and its policy context. 



An educational intervention's effectiveness is judged by whether it produces positive 
outcomes for students, with the randomized controlled trial (RCT) as a valuable tool for 
determining intervention effects. However, the intervention-as-implemented in an experiment 
frequently differs from the intervention- as -designed, making it unclear whether unfavorable 
results are due to an ineffective intervention model or the failure to implement the model fully. 
It is therefore vital to assess fidelity of implementation and, where possible, to incorporate 
fidelity data in the analysis of outcomes. 

Purpose / objective / research question / focus of study: 

Description of what the research focused on and why. 



Our objectives are to 1) elaborate a five-step procedure for fidelity assessment, 2) 
describe the advantages of assessing fidelity with this approach when conducting experiments 
testing educational interventions, and 3) use examples based on published studies to illustrate 
how this procedure can be applied. 

Setting: 

Description of where the research took place. 

N/A 

Population / Participants / Subjects: 

Description of participants in the study: who (or what) how many, key features (or characteristics). 



We examine five examples of experiments that tested educational interventions in a range 
of subject areas (e.g., mathematics, reading, science, behavior) and grade levels (e.g., preschool, 
elementary, high school). 

Intervention / Program / Practice: 

Description of the intervention, program or practice, including details of administration and duration. 



Fidelity is the extent to which an intervention is implemented as intended. Fidelity 
assessment describes the cause-and-effect sequences inside the experimental “black box” that 
collectively make up the implementation of the intervention. By assessing fidelity, we determine 
the extent to which this sequence occurred as originally envisioned, thus assessing construct 
validity and providing a basis for generalization. While not always possible, the ideal fidelity 
assessment includes five steps (Cordray, 2007): 1) describe the intervention change and logic 
models; 2) create indices to measure the fidelity of the implemented intervention to constructs 
identified in the models; 3) determine the reliability and validity of indices; 4) combine indices 
in the analysis; and 5) link fidelity measures to outcomes. 
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Research Design: 

Description of research design (e.g., qualitative case study, quasi-experimental design, secondary analysis, analytic 
essay, randomized field trial). 



As a foundation for our paper, we discuss the limitations of the intent-to-treat 
experimental model for explaining effects, and the advantages of using fidelity measures to look 
inside the black box. We describe the five-step assessment process and the rationale for each 
step. We then demonstrate the procedure's application using several published studies that 
followed one or more of these steps, illustrating how this approach can be applied practically and 
the benefits it yields. 

Data Collection and Analysis: 

Description of the methods for collecting and analyzing data. 



Studies examined are drawn from a pool of 526 articles across subjects and grade-levels, 
identified through a review based on O'Donnell's (2008) methodology. The search employed 14 
databases, including ERIC, PsycINFO, JSTOR, Dissertation Abstracts, and Google Scholar. We 
narrow the scope of articles to peer-reviewed studies involving experiments. 

Findings / Results: 

Description of main findings with specific details. 



We find that, while most analyses of fidelity in educational studies include one or more 
steps in the five-step fidelity assessment process, the typical study does not include all steps 
and/or executes steps incompletely. As a result, the usefulness of fidelity data for interpreting 
experimental results is diminished. We demonstrate that a more complete assessment of fidelity 
in such cases would lead to a more meaningful analysis that interprets experimental results in 
light of actual implementation. 

Conclusions: 

Description of conclusions and recommendations based on findings and overall study. 



We conclude that the fidelity assessment procedure can be followed for a range of 
intervention types and subjects. Furthermore, this approach leads to more complete fidelity 
assessment and thus to better understanding of construct validity and generalizability. By 
advancing these arguments and providing examples, we encourage broader understanding of the 
importance of fidelity assessment, more complete assessment, and improved methods for 
assessment. 
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Appendices 

Not included in page count. 
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Appendix B. Tables and Figures 
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